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ABSTRACT 



The Texas "Families First" project was started to accomplish 
the following: (1) strengthen the capacity of local family literacy projects 

to design, support, and administer high quality programs by leveraging 
resources from several agencies in the state; and (2) finalize the indicators 
of program quality (IPQs) in early childhood education for Even Start family 
literacy programs. The three institutions of higher education involved in the 
project during the first year identified instruments for assessment in early 
childhood education and provided opportunities for staff in Head Start and 
Even Start programs to learn to use the instruments and to conduct follow-up 
meetings with Even Start programs . Findings from the meetings included the 
following: (1) only 51 percent of Even Start programs were represented at the 

focus group meetings; (2) although participants learned a great deal about 
the instruments, they did not feel they were able to administer them properly 
without more training; (3) three assessments were identified as used most 
often in Even Start; (4) participants identified strengths and weaknesses for 
the five instruments demonstrated in the training meetings; and (5) some 
programs plan to add at least one of the instruments to their assessment 
package . Recommendations were made for improvements in the timing of 
assessment training, cost of assessment instruments, standardization of 
assessment across the state, program accountability, input from early 
childhood specialists in the development of the IPQs, accountability based 
upon factors beyond a local program's control, and standardization versus the 
unique needs of each program. (KC) 
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Executive Summary 



The Texas “Families First” project was initiated to (1) strengthen the capacity of local 
family literacy projects to design, support, and administer high quality programs through 
leveraging resources from several agencies in the State, and (2) finalize the indicators of 
program quality (IPQs) in early childhood education for Even Start family literacy 
programs. Since the IPQs for adult education had been finalized in previous years, one 
focus of the project was to develop those that were to be used in early childhood 
education. 

Three institutions of higher education were involved in the project during the first year; 
The Center for Improving the Readiness of Children for Learning and Education 
(CIRCLE), The Texas Center for Adult Literacy and Learning (TCALL), and The Texas 
Family Literacy Center, University of Texas at Austin. The role of CIRCLE, University 
of Texas at Houston Health Science Center, was twofold. First, CIRCLE staff identified 
appropriate instruments for assessment in early childhood education. Second, CIRCLE 
staff provided opportunities for staff members in Head Start and Even Start programs in 
the State to become familiar with those instruments and to learn the basic requirements 
for their proper administration. Those opportunities were provided through a series of 
training meetings in the State during the summer of 2000. 

The role of TCALL was twofold as well. The first role was to attend some of the 
CIRCLE training sessions mentioned above. The second role was to conduct follow-up 
meetings with Even Start programs in the State. Input from those attending the meetings 
provided information regarding (1) the perception of the effectiveness of the training held 
by CIRCLE, (2) the status of frequency of use of the instruments, and (3) the extent to 
which programs plan to use the instruments in the future. 

Findings from the meetings included: 

1 . Only 51% of Even Start programs in the state were represented at the focus group 
meetings. 

2. Participants learned much about the instruments. However, they do not feel 
adequate to administer them properly without more “in-depth” training. 

3. Although there was some familiarity with all of the instruments, the participants’ 
responses revealed that Preschool Language Survey, Third Edition (PLS-3), the 
Peabody Picture Vocabulary Test-Third Edition (PPVT-III), and the Bayley 
Infant Neurodevelopmental Screener (BINS) are the most-widely used early 
childhood assessments in Even Start in Texas. 

4. Participants identified strengths and weaknesses for all of the five instruments 
demonstrated in the CIRCLE training. 
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5. Some programs, especially those that are newly-funded, plan to add at least one 
of the instruments to their assessment package. 

6. There were several recommendations for the Texas Education Agency in regard 
to future training for assessment in Even Start programs. Some of these 
recommendations addressed: 

a. Timing of assessment training 

b. Cost of assessment instruments 

c. Standardization of assessment across the state; and 

d. Program accountability being based mostly upon testing 

7. There were also several recommendations in regard to the IPQs, including: 

a. Input from Early Childhood specialists in the development of the IPQs 

b. Accountability based upon factors that are beyond a local program’s 
control 

c. Standardization vs. the unique needs of each program 
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Introduction 



Almost 30 percent of Texas children live in poverty and almost half of all public school 
students zire considered to be economically disadvantaged. The Texas Adult Literacy 
Study also documents that 27 to 28 percent of all Texas adults function in the lowest level 
of literacy and another 25 to 27 percent function in the second lowest level. 

Family literacy is a model that can expand the promise of public education from early 
childhood to adulthood. Strengthening and expanding family literacy services is 
imperative for Texas if education is to fulfill that promise. 

The objectives of the Families First Initiative zire: 

1. To strengthen Texas family literacy projects through development, 
implementation and refinement of Indicators of Program Quality (IPQs). 

As bases for evaluating Even Start project program performance and improvement over 
time, development and implementation of Indicators of Program Quality will strengthen 
family literacy services. The IPQs will provide a framework for local family literacy 
projects to tell how well they are doing, and where they need to improve by providing 
information on the performance of each p articipant (children and parents). The IPQs will 
also serve as a framework for measures of continuous improvement over time so that 
family literacy projects can track sustained success or needed areas of improvement. 

2. To strengthen Texas family literacy through implementation of professional 
development and technical assistance for family literacy staff based upon the best 
available research on emerging literacy, language development, and reading 
instruction. 

Professional development will be based on the best available research on emerging 
literacy, language development, and reading instruction, especially for families who are 
limited English proficient, migrant, or homeless, as well as adults and children with 
disabilities. A professional development plan will be structured to assist family literacy 
projects as they build their own local capacity through a “trainer of trainers” approach 
and will encourage family literacy projects to develop their own training teams. 

3. To expand family literacy services through coordination and integration of 
resources of the programs represented on the consortium. 

The Family First Consortium includes representatives from: 

• The Texas Education Agency (Title I, Parts A, B, C, and E, the Adult Education 
and Family Literacy Act; state funded preschool programs, and state funded adult 
literacy programs; Individuals with Disabilities Education Act; Reading 
Excellence Act); 

• The University of Texas State Head Start Collaboration Project; 
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• The Texas Department of Human Services (TDHS, the state IV-A 
agency/eligibility for Temporary Assistance to Needy Families (TANF)); 

• The Texas Workforce Commission (Work First and childcare services for TANF 
Recipients); 

• The Governor’s Business Council; 

• The Texas Family Literacy Center at The University of Texas at Austin; 

• The Texas Center for Adult Literacy and Learning at Texas A&M University; 

• Del Valle ISD, Northside ISD, Harris County Department of Education, and 
Child, Inc., 

• The Texas Head Start Association; and 

• The Texas Association for Literacy and Adult Education (TALAE). 

Amendments to the Reading Excellence Act of 1998 require states to develop Indicators 
of Program Quality (IPQs) for evaluating Even Start programs. The legislation stipulates 
that each state's IPQs will include the following indicators, which address progress of 
eligible child participants: 

• Improvement in ability to read on grade level or reading readiness; 

• School attendance; 

• Grade retention and promotion; and 

• Such other indicators as the State may develop. 

The Texas Education Agency was awarded a two-year federal grant under the Even Start 
Family Literacy Statewide Initiative Grant program to assist the State in strengthening 
family literacy services through the development and implementation of IPQs. One of 
the Texas Families First Initiative priorities is the development of IPQs for Even Start 
programs with respect to eligible participants who are children, ages 0-7. Texas intends 
to develop IPQs that will reflect the child's developmental progress and emergent literacy 
as the child moves into preschool and elementary education. 

The Center for Improving the Readiness of Children for Learning and Education 
(CIRCLE) at the University of Texas Health Science Center at Houston is involved in the 
identification of age-appropriate evaluation tools to be used to assess specific aspects of 
the State Head Start preschool program. The Families First Project intends to leverage 
the outcomes of CIRCLE'S Head Start research for use in developing IPQs in regard to 
children ages 3-to-5 years in Texas family literacy programs. CIRCLE has identified 
age-appropriate assessment tools for the family literacy early childhood component, ages 
0-3, particularly in the area of developmental progress, as part of the Families First 
project. 

Those assessment instruments are: 

• Peabody Picture Vocabulary Test (PPVT) III 

• TVIP (Spanish edition of the PPVT) 

• Expressive Vocabulary Test (EVT) 

• Preschool Language Scale -3, English and Spanish Editions (PLS-3); 
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• Developing Skills Checklist/Lista de Destrezas en Desarrollo (DSC); and 

• Bayley Infant Neurodevelopmental Screener (BINS) 

Through a series of regional workshops in the Summer of 2000, CIRCLE provided the 
opportunity for training in the use of these six assessment instruments to Even Start 
family literacy program staff in the State. "Assessment teams" of 2-3 people from some 
Even Start projects participated in learning the appropriate uses of the early childhood 
assessments. Training sessions were held in Fort Worth, Houston, Dallas and Corpus 
Christi. 

As a follow-up to the training that CIRCLE offered, staff at the Texas Center for Adult 
Literacy and Learning (TCALL), Texas A«&M University-College Station, conducted 
meetings with “focus groups” at six regional meetings in the Fall of 2000. Participants 
included Even Start staff who had attended the CIRCLE training as well as those who 
had not attended or were newly-funded Even Start programs. TCALL acquired input 
from those who had attended the training in regard to development of the actual indicator 
statements, feedback on the assessment instruments identified by CIRCLE, and 
professional development needs related to the IPQs. A report of the information from the 
work that TCALL coordinated is foimd on pages 11. 



Project Procedures 

According to the geographical distribution of Even Start projects in the State, TCALL 
conducted a series of “focus group” meetings in Huntsville, Dallas, Houston, Edinburg, 
San Antonio, and Austin to learn: 

1 . the extent to which the six assessment instruments are being used; 

2. the concerns from Even Start programs about the instruments; and 

3. information about the content of the IPQs. 

A letter was mailed to each Even Start program regarding the most convenient time for 
the program staff to attend their focus group meetings. The selection for focus group 
meeting dates was based on the feedback from those program staff 

After the dates were selected, letters were sent out to Even Start programs inviting them 
to attend the focus group meetings. Each meeting was held from 9 A.M. to 12 P.M. in a 
regional Education Service Center. It began with an introduction of the purpose of the 
meeting, followed by participants’ comments on the issues regarding Performance 
Indicators, assessment instruments identified by CIRCLE, plus assessment instruments 
that are being used in Even Start programs but were not identified during the CIRCLE 
training. 
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TABLE 1 
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Site 


Huntsville 


Dallas 


Houston 


Edinburg 


San Antonio 


Austin 


Total 


Date of 
meeting 


Oct. 27 
(Fri.) 


Nov. I 
(Wed.) 


Nov. 3 
(Fri.) 


Nov. 7 
(Tue.) 


Nov. 9 
(Thu.) 


Nov. 17 
(Fri.) 




Number of 
programs 
invited 


6 


II 


13 


II 


13 


II 


65 


Number of 
participants 


7 


I 


18 


15 


7 


14 


55 


Number of 
programs 
represented 


2 


I 


10 


10 


5 


7 


35 



Description of Assessment Instruments Identified by CIRCLE 

Peabody Picture Vocabulary Test-Third Edition (PPVT-III) 

The PPVT-III is a measure of listening comprehension for spoken words in steindard 
English and a screening test of verbal ability. The PPVT-III offers standard scores, 
percentiles, age equivalents, and other related information. It is suitable for children from 
2.6 years of age and older. This individually administered, norm-referenced instrument is 
offered in two parallel forms -IIIA and IIIB- for reliable testing and retesting. 

The test-retest reliability of PPVT-III is .91 to.94. As to validity, the PPVT-III has an 
average correlation of .69 with the OWLS Listening Comprehension scale and .74 with 
the OWLS Oral Expression scale. Its correlations with measures of verbal ability are; .91 
(WISC-III VIQ), .89 (KAIT Crystallized IQ), and .81 (K-BIT Vocabulary). The 
Technical References supplement compares PPVT-III scores of eight special populations 
(speech impaired, language delayed, language impaired, mentally retarded [child and 
adult], reading disabled, hearing impaired, and gifted) with demographically matched 
control groups. 

Test de Vocabulario en Imagenes Peabody (TVIP) 

The TVIP is a measure of receptive vocabulary for Spanish-speaking children and 
adolescents. It measures Spanish vocabulary based on the widely used PPVT. The TVIP 
is suitable for children from 2.5 to 18 years of age. The TVIP can be used for the 
following purposes: 

1 . evaluating the language development of Spanish-speaking preschool children. 

2. screening Spanish-speeiking children entering kindergarten or first grade. 

3. determining the more effective language of instruction for bilingual children. 

4. evaluating the Spanish vocabulary of older students. 
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Expressive Vocabulary Test (EVT) 

The EVT is a measure of expressive vocabulary and word retrieval for Standard 
American English. The EVT is suitable for children from 2 years of age and older. It 
offers age-based standard scores, percentiles, test-age equivalents, and other related 
information. It is an individually administered, norm-referenced test of expressive 
vocabulary and word retrieval. The EVT and PPVT-III were standardized on the same 
population of 2,725 examinees ranging in age from 2-6 to 90+. This conforming lets its 
users make direct comparisons of receptive and expressive vocabulary. The national 
sample was stratified to match the most recent U.S. Census on gender, race/ethnicity, 
region, and socioeconomic status. 

The EVT features: 

1. quick administration and scoring 

2. untimed administration rules 

3. age-appropriate start items plus basal and ceiling rules 

4. full-color stimulus pictures 

5. portable testing easel 

The EVT has a high degree of internal consistency. Split-half reliabilities range from .83 
to .97 with a median of .91 . Test-retest studies with four separate age samples resulted in 
reliability coefficients ranging from .77 to .90, indicating a strong degree of test stability. 

Preschool Language Scale -3 (PLS -3) 

The PLS-3 measures a broad range of receptive and expressive language skills. The 
PLS-3 is suitable for children from birth through 6 years of age. The norm-referenced 
PLS-3 provides standard scores, percentile ranks, and age equivalents for auditory 
comprehension and expressive communication, and a total language composite. The 
normative sample of the PLS-3 included more than 1200 children throughout the United 
States. The PLS-3 focuses on the development of syntax, morphology, vocabulary, 
vocabulary concepts, and integrative thinking skills (e.g. classification, word definitions). 
The PLS-3 tasks are ordered to reflect children's acquisition of sequential developmental 
milestones in language. 

Data provided in the Examiner’s Manual shows evidence of reliability. The tasks in 
PLS-3 are homogenous (internal consistency), PLS-3 scores are dependable and stable 
- across repeated administrations (test-retest reliability), and scoring is objective and 
consistent across examiners (inter-rater reliability). 

The same manual also provides data that shows evidence of validity. Research conducted 
with the PLS-3 provides evidence that it offers a thorough and balanced sample of 
language behaviors (content validity), it consistently differentiates children who are 
language disordered from children who are not (construet validity), and its scores are 
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highly correlated with scores obtained from other valid measures of language ability 
(concurrent validity). 

Developing Skills Checklist (DSC) 

The DSC and its Spanish Version, Lista de Destrezas en Desarrollo (La lista), are suitable 
for Pre-K and K populations. Both measures assess pre-reading, mathematics, social and 
emotional development, fine and gross motor skills, and print and writing concepts. The 
test measures the skills and concepts that help determine appropriate instruction for 
individual £is well as groups. The test is individually administered and gives the 
assessment staff opportunity to observe children in the natural environment of the 
clEissroom. It is e£isy to administer and score. "La Lista" tests the developing skills of 
Spanish-speaking children in Bilingual, English as a Second Language, and Title I 
Programs. La Lista uses standard Spanish common to all dialects and incorporates 
positive references to Spanish Speaking cultures. 

Bayley Infant Neurodevelopmental Screener (BINS) 

The BINS is designed to identify infants between the ages of 3 and 24 months who are 
developmentally delayed or have neurological impairments. It is appropriate for different 
developmental ages. The BINS includes a subset of items from the Bayley Scales of 
Infant Development-Second Edition as well as items that assess muscle tone and quality 
of movement. It is a tool for screening infants to identify them for further diagnostic 
testing. The BINS emphasizes a process approach by considering how an ability is 
expressed, rather than simply whether the ability is exhibited. 



Data Acquired by TCALL 

Comments on Training Held by CIRCLE 

Participants attending the TCALL-conducted Fall 2000 focus group meetings offered 
these comments regarding the CIRCLE-led Summer 2000 training sessions. 

• The training sessions provided a good overview of the assessment instruments. 
However, participants do not feel comfortable about their abilities to administer 
the test accurately. 

• There seemed to be differences in the quality of facilities available, differences in 
the trainers as well as the information that different trainers provided. For 
example, some participants at the CIRCLE trainings were told about video 
certification process for the Bayley Infant Neurodevelopmental Screener, but 
some were not. It should be noted that the trainers were sometimes operating 
under difficult circumstances. At one sight, almost twice as many Head Start 
participants attended as were expected and the trainer had to find another site at 
the “last minute”. That also constrained the adequacy of materials available. 
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• Most participants think that more intensive training is needed on administering 
the assessment instruments. They would like to see videos of the various 
assessments being administered. They also think it is important for them to 
receive training of increased intensity and quality (similar to the BEST and TABE 
training in adult education) so that they are confident to administer the tests. 

Comments on Assessment Instruments Identified by CIRCLE 

Remarks concerning each of the assessment instruments that CIRCLE identified in its 

Summer training sessions are provided below. 

The Peabody Picture Vocabulary Test-Third Edition (PPVT-III) 

• Advantage: 

a. The PPVT-III is quick, fast, and easier to administer and score than other 
assessment instruments. 

• Disadvantages: 

a. The portion of Word Recognition is a bit "limited". The participants indicated 
that they want a test that addresses more. 

b. Some pictures do not communicate the meaning intended very well. 

c. Acquiring the basal and "top" levels requires too much time— the standards are 
often not realistic of the child’s real ability. 

d. Attention span of the children can be a problem. 

e. It is quick and somewhat accurate for baseline assessment, but not for exit 
assessment. 

f School counselors sometimes use it for diagnosis purposes. It can cause 
problems when two different results are reported from the counselors and 
Even Start program staff 

The Test de Vocabulario en Imasenes Peabody (TVIP) 

• Advantage: 

a. The TVIP is quick, fast, and easier to administer and score compare to other 
assessment instruments. 

• Disadvantages: 

a. There was not enough training provided for its users to administer and 
interpret the test. 

b. It does not seem accurate. 

c. The TVIP is not equivalent to the PPVT-III; therefore, no comparison 
between TVIP scores with PPVT-III scores can be made. 
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The Expressive Vocabulary Test (EVT) 



• No Even Start program staff members that attended the focus group meetings 

were using the EVT during 2000. 

The Preschool Lansuaze Scale -3 (PLS -3) 

• Advantages: 

a. It is a good instrument for children 1 8 months to 3 years of age. 

b. The PLS -3 gives teachers a good idea about what to do in the classroom, and 
they can translate PLS -3 scores to classroom activities. 

c. Age is a concern, but the PLS -3 has a chart for conversion. 

d. It is often chosen due to familiarity. 

• Disadvantages: 

a. Scoring is time-consuming. 

b. It seems to score the students lower than they should be scored. 

c. Some words don't translate well into Spanish. 

d. It requires a long attention span— 45 minutes, which can be a problem when 
testing young children. 

e. Some content is not relevant to what children do and can relate to. 

f Not all questions are related to early childhood instruction. 

g. The PLS -3 is suitable for English children, but not for non-English speaking 
children. 

h. Using both English and Spanish versions is more problematic. 

i. The PLS -3 is used as an assessment tool by Even Start programs and as a 
diagnostic tool by Special Education program. Problems occur when two 
different results are reported for the same children. 

j. Most participants feel a need for more training before being comfortable using 
it, although they are using it now. 

The Developing Skills Checklist (DSC) 



• Disadvantages: 

a. It is not age-appropriate for the population some Even Start programs serve. 

b. It is not appropriate to compare the scores from English and Spanish versions 
of the DSC because they are not equivalent. 
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The Bayley Infant Neurodevelopmental Screener (BINS) 



• Advantages: 

a. Some feel that adequate training on the BINS can be obtained from the 
manual when the video, which can be ordered, supplements the manual. The 
video is very explicit for training. 

b. It provides a score for those below two years of age. 

c. It is easy to use and score. 

d. The BINS is easier to administer than PLS -3, especially for testing non- 
verbal children. 

• Disadvantages: 

a. The packet of materials needed for testing can be a burden. 

b. Children are not doing anything. It is a “passive” instrument and difficult to 
administer. 

c. Many program staff members are not comfortable to make a judgment based 
on observations. 

Comments on Assessment Instruments Not Identified by CIRCLE 

The participants attending the focus group meetings provided comments on several 
assessment instruments that CIRCLE did not identify. They are Denver II, Ages & 
Stages Questionnaire (ASQ), PALM, Rossetti Infant Toddler Language Scale, IDAS, I- 
pass from Kaplan, Brigance Screens, Oral language proficiency test, IDEA Oral 
Language Proficiency Tests (IPT), and CEDEN. When information could be located, a 
brief description and comments for each assessment instrument are provided below. 

Denver II: 



• Description: 

Denver II is designed to be used with children between birth and six years of age 
and is administered by assessing a child's performance on various age-appropriate 
tasks. The test is valuable in screening asymptomatic children for possible 
problems, in confirming intuitive suspicions with an objective measure, and in 
monitoring children “at risk” for developmental problems, such as those who have 
experienced perinatal difficulties. 

• Comments: 

a. It tells how much delay a child is at certain months of age. 

b. It is very practical. 

c. It shows what a child should be doing at various ages. 
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Ages & Stases Questionnaire (ASQ) 



• Description: 

The ASQ is a screening tool used on home visits, mailings to families and 
telephone calls. The provider examines the child's developmental levels through 
questions answered by the caregivers/parents. 

• Comments: 

a. It observes daily activities. 

b. Some programs use Ages & Stages every three months for progress 
assessment. 

c. It can help parents assess children. 

d. There is a Spanish and an English version as well as a training video. The test 
developers also provide training. 

e. It could be used during home visits. 

PALM 



• Description: 

No information was found by TCALL staff. 

• Comments: 

a. It measures social and emotional development of a child. It is not a reading 
readiness test, but it might be useful for Even Start programs. 

Rossetti Infant Toddler Language Scale 

• Description: 

This is a criterion-referenced instrument designed to provide the clinician with a 
comprehensive tool that assesses the preverbal and verbal aspects of 
communication and interaction in young children. It measures interaction- 
attachment, pragmatics, gesture, play, language comprehension, and language 
expression. The results from this assessment tool reflect the child's mastery of 
skills in each of the areas assessed at three-month intervals. This scale promotes 
the family's role as a full partner and may be administered in the home, diagnostic 
center, school, clinic, or hospital setting. 

• Comments: 

a. Rossetti Infant Toddler Language Scale requires interaction which can 
produce positive or negative reactions from the child. 

b. It is not a developmental assessment instrument. 

c. It is more open and contextual than other instruments for age-specific groups. 

d. It includes social-emotional measures that are important to some coordinators. 
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IDAS 



• Description: 

No information was found by TCALL staff. 

• Comments: 

a. It acquires mostly observational data which can be subjective. 

b. It covers 6 important skills/developmental areas, and is very appropriate to 
measure what children actually do. 

c. It can be used regularly to show parents the progress of their children. 

I-pass 

• Description: 

No information was found by TCALL staff. 

• Comments: 

a. It is a good test for young children (0-3 years of age) 

Brigance Screens 

• Description: 

Brigance Screens are designed to assist teachers with program plarming, indicate 
developmental problems— language, learning, or cognitive delays— and identify 
children who have academic talent or intellectual giftedness. It samples children's 
skills in the following areas: fine and gross motor, general knowledge, language, 
pre-academic/academic, graphomotor development. It also includes a set of at- 
risk guidelines for use in prevention programs to identify children in need of 
prompt referral. 

• Comments: 

a. It is used for pre-and-post test data in some Even Start programs. 

Oral Lansuase Proficiency Test 

• Description: 

No information was found by TCALL staff. 

• Comments: 

a. It is used for pre-K and K 

b. It is for oral language screening, not a diagnosis tool. 

c. It helps to determine a child’s fluency level in speaking English. 
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IDEA Oral Lan^age Proficiency Tests (IPT) 



• Description: 

The IPT is available for pre-kindergarten through grade 12 students. It is written 
in both English and Spanish. The IPT oral tests provide Non-, Limited, and 
Fluent English and Spanish Speaking designations. All IPT tests are written and 
normed to meet the standards of the American Psychological Association, and the 
IPT I«feII-Oral Spanish Tests are written and normed specifically for Spanish- 
speaking students. 

• Comments: 

a. It is easy to administer. 

b. It is accurate. 

c. It is also used for placement for bilingual children, and ESL. 

CEDEN 

• Description: 

No information was found by TCALL staff. 

• Comments: 

a. It is a checklist designed to address children from 0 to 5 years of age. 

b. It is very similar yet covers more developmental areas than PLS-3. 

Early Lansuase Milestone Scale, Second Edition 

• Description: 

This language screening tool assesses language development from birth to 36 
months of age and intelligibility of speech from 1 8 to 48 months of age. It has 
three subtests: Auditory Expressive Language Development, Auditory Receptive 
Language Development, and Visual Language Development. It is sensitive to 
various causes of speech or language delay, including hearing loss, mental 
retardation, autism, dysarthria, and stuttering. It does not yield a specific 
developmental diagnosis. It is designed to identify or quantify language delay in 
very young children so that they may receive care as promptly as possible. 

• Comments: 

No comment was made. 
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Comments on Performance Indicators Provided by TEA 

• An indicator for Early Childhood Development should be included in any 
program evaluation. 

• TEA needs to inform the programs what assessment instruments will be required 
much earlier than in the past so that staff may be properly and thoroughly trained 
to administer the tests in a timely marmer. 

• TEA should adopt a policy for each program to use instrument(s) appropriate for 
its population. 

• Even Start programs have no control over grades, retention, and promotion of 
children in pre-K or higher grade-levels. Should they still be held accountable? 

Overall Concerns and Questions 

Resardins the Division of Adult and Community Education, Texas Education Agency: 

• The expense increased in purchasing assessment instruments is a concern of Even 
Start programs. 

• It will be beneficial to know what all Even Start programs are using for 
assessment, and to exchange information among the program staffs. 

• If every program needs to follow the same indicators, why not require every 
program to use the same instniment(s)? 

Regarding Assessment Instruments 

• Assessment instruments identified by CIRCLE only look at part of the child. They 
do not look at the whole child. 

• Should evaluators administer the tests? 

• What should a staff do when a program has Spanish-speaking students and non- 
Spanish speaking teachers? 

• When children are neither English-proficient nor Spanish-proficient, what kind of 
instrument should be used? 

• Can college students who are learning to administer tests partner with Even Start 
programs so that they could give the tests? 

Regarding Training 

• Are there videos available for all of the assessment instruments for individual 
training? 

• Who will provide the training? What are the qualifications required for those who 
are to administer the tests? 

• Early Childhood Intervention (ECI) is funded throughout the state. They serve 
diagnosis purposes and early intervention for children with developmental delay. 



They conduct home visits and help peirents to help language delayed children. 

Cein ECI train people to give tests? Can they give training on a group basis? 

• Can Even Start programs as a group ask test/assessment instrument publishers to 
send people to train Even Start personnel? If a sufficient number of people need 
to be trained, then publishers will sometimes provide needed training. 

• Where can Even Start programs observe the instruments actually being 
implemented? 

Survey Results 

The staff at TCALL designed the Families First Even Start survey which was distributed 
to every participant in each focus group meeting. The survey contained questions related 
to the use of the assessment instruments identified by CIRCLE, the use of assessment 
instruments not identified by CIRCLE, and participants' feedback on the focus group 
meetings held by TCALL. Forty-eight participants completed the survey that was 
administered at the conclusion of each focus group meeting. 



TABLE 2 



Number .of Participants Who Completed the Survey 


Site 


Huntsville 


Dallas 


Houston 


Edinburg 


San 

Antonio 


Austin 


Total 


Number of 
participants 


7 


1 


13 


10 


7 


10 


48 



The Use of Assessment Instruments Identified by CIRCLE 

Among the six assessment instruments identified by CIRCLE, twelve (25%) participants 
indicated that they are using Peabody Picture Vocabulary Test (PPVT-III), 5 (10%) are 
using TVIP (Spanish edition of the PPVT), 4 (8%) are using Expressive Vocabulary Test 
(EVT), 30 (63%) are using Preschool Language Scale-3 (PLS-3), 5 (10%) are using 
Developing Skills Checklist (DSC), and 10 (21%) are using Bayley Infant 
Neurodevelopmental Screener (BINS). Table 3 shows the frequency of use by focus 
group participants of each assessment instrument identified by CIRCLE. 



TABLES 



The Frequency of Use by Focus Croup Partidpants of the 
Assessment IttstnimentsMiM^^ 


Assessment 

Instrument 


PPVT-III 


TVIP 


EVT 


PLS-3 


DSC 


BINS 


Number of 
participants 


12 


5 


4 


30 


5 


10 



Twenty-four (50%) participants indicate that the people in their programs administrating 
the assessment instruments were trained, 12 (25%) participants do not think that the 
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people in their programs administrating the assessment instruments were appropriately 
trained, and 12 (50%) did not respond to this question. Table 4 shows those data. 



TABLE 4 



Bow Wert Pmpk Tmine4 to V8B 

. , j ,:;ii=,2i’* " ' , 


Trained by 


District 


Manua 

1 


CIRCLE 


College/ 

University 

courses 


Previous 

Job 


Workshops vs. 
Training sections 


Number of 
participants 


10 


3 


7 


0 


0 


3 



* Some participants were trained in multiple ways. 



In the survey, the participants who have used the assessment instruments identified by 
CIRCLE responded to the question regarding user-friendliness of the assessment 
instruments. Table 5 shows the results. 



TABLES 



Responses Rejgarding User'-Priendliness of Assessment Instramenits 




Very User- 
friendly 


User-friendly 


Somewhat 

User-friendly 


A Little 
User-friendly 


Not User- 
friendly 


PPVT-III (n=10) 


1 


5 


3 


1 


0 


TVIP (n=5) 


0 


4 


0 


1 


0 


EVT (n=3) 


0 


2 


0 


1 


0 


PLS-3 (n=26) 


1 


11 


10 


4 


0 


DSC (n=5) 


1 


1 


0 


3 


0 


BINS (n=8) 


3 


2 


2 


1 


0 



In the same survey, the participants who have used the assessment instruments identified 
by CIRCLE responded to the question regarding the overall satisfaction of these 
assessment instruments. Table 6 shows the results. 



TABLE 6 



Responses Regarding Qvei^ Satis&cticm of Ags^^isment 




Very 

Satisfied 


Satisfied 


Somewhat 

Satisfied 


A Little 
Satisfied 


Not 

Satisfied 


PPVT-III (n=ll) 


0 


5 


5 


1 


0 


TVIP (n=5) 


0 


3 


1 


1 


0 


EVT (n=3) 


0 


2 


0 


1 


0 


PLS-3 (n=26) 


1 


8 


12 


5 


0 


DSC (n=5) 


1 


1 


0 


3 


0 


BINS (n=9) 


3 


3 


2 


1 


0 
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The Use of Assessment Instruments Not Identified by CIRCLE 



The participants are also using other assessment instruments. Those instruments in order 
of frequency of use are as follows: Denver II, Early Language Milestone Scale, IDEA 
Oral Language Proficiency Tests (IPT), Ages & Stages Questionnaire (ASQ), Brigance 
Screens, Home Screening Questionnaire Checklist, Rossetti Infant Toddler Language 
Scale, Parent-teacher Survey, CEDEM, Baylor University Observational Checklist, 
Partners for Learning, PSI, Home Gown Checklist, IDS, and Infant Development 
Assessment Scale. Table 7 shows the most commonly used assessment instruments, and 
the number of participants who are using or used them. 



TABLE 7 



Ibe Rt^qiiency of U# by focu» Group 


rticiponts of 
ifCaiRoyE 


Assessment 

Instrument 


Denver 

II 


Early 

Language 

Milestone 

Scale 


IPT 


Brigance 


ASQ 


Home 

Screening 

Questionnaire 

Checklist 


Rossetti Infant 
Toddler 
Language 
Scale 


Number of 
participants 


11 


4 


4 


2 


2 


2 


2 



Other Related Information 

Twenty-four (50%) participants indicated that the focus group meetings held by TCALL 
helped them very much, 10 (21%) think that the meetings helped them much, 9 (19%) 
participants think that the meetings helped them some, 4 (8%) think that the meetings 
help them a little, and 1 (2%) did not respond to the question. 

After the focus group meetings were initiated, TCALL staff added the following question 
‘‘‘'Did you attend the training section on Early Childhood Assessment Instruments?” Of 21 
participants, 6 (29%) responded yes and 15 (71%) responded no. 

Twenty-one participants were asked the question “Since the training (held by CIRCLE), 
what instruments have you purchased or planned to purchase?” Table 8 shows the 
results. 



TABLE 8 



Inshrunionite Furciiasei or Hoimod to f M 
the Training by CmCLE 

m-0^ 




Instrument 


PPVT-III 


TVIP 


EVT 


PLS-3 


DSC 


BINS 


Others 


Number of 
participants 


4 


3 


0 


5 


2 


2 


1 



Some participants purchased or p 



armed to purchase more than one assessment 



instrument after attending the training by CIRCLE. 
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Twenty-one participants were asked the questions ‘‘‘‘After today's (focus group) meeting, 
what instruments do you plan to purchase?’’’’ Table 9 shows the results. 



TABLE 9 




Instrument 


PPVT-III 


TVIP 


EVT 


PLS-3 


DSC 


BINS 


Others 


Number of 
participants 


0 


0 


1 


5 


1 


1 


2 



* Some participants planned to purchase more than one assessment instrument after 
attending the focus group meetings by TCALL. 



Overall Recommendations 

Regarding the Division of Adult and Community Education, Texas Education Agency: 

Participants’ Recommendations 

• Have final proposed measures (and IPQs) include Early Childhood Specialists' 
recommendations. 

• Follow up any future focus group meetings with a local contact at TEA to provide 
intensive training for assessment and/or other activities. An attempt to re-activate 
the regional cluster groups should also be initiated. 

• The State needs to be consistent in assessment instruments across different grants 
and programs so there is no need to test the child more than once. This has 
become a problem with counselors in some school districts. 

• There is a need for an Even Start Early Childhood Consultant at TEA. 

TCALL Staffs Recommendations 



• Hold another round of cluster meetings after the 2001 post-testing (June or 
August) and make it mandatory that every program be represented. The focus of 
the meetings should be to initiate some kind of standard assessments while 
allowing programs to assess for their individual needs. 

• Clarify if the IPQs will be for children's language skills in English, or language 
skills in general. 
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• Make sure that the Requests for Application (especially for “renewal grants”) and 
their subsequent approval are announced early enough so that programs can avoid 
losing experienced staff due to the lateness of funding. 

Regarding Assessment Instruments: 

Participants Recommendations 

• Have partnership with local university to have someone administer the tests. 

• Find an instrument that simply measures vocabulary and language by observing 
and recording all words said within a certain time period. 

• Observe the child frequently in her/his natural environment, not depending upon a 
one-day performance. 

• Take the teacher's evaluation into consideration because he or she knows what a 
child can do much more than what a test will show. 

• Develop assessment instruments that are developmentally appropriate and target 
the audience. 

TCALL Staffs Recommendations 



• Focus on assessments that are available in both English and Spanish. 

• Identify assessment instrument(s) for accessing reading readiness. The six 
instruments used in the training focus more on language development, whereas 
reading readiness is one of the elements in the performance indicators. 
Otherwise, change the performance indicator to reflect language development. 

Regarding Training: 

Participants’ Recommendations 

• Provide more training on assessment instruments so that program personnel can 
be confident in what they tell parents, and in any referrals they might make for 
children who are developmentally delayed. 

• Provide competent technical assistance/follow-up after the training has been 
completed. 
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TCALL Staffs Recommendations 



• Provide intensive training for local Even Start programs in regard to appropriate 
methods for using assessment iristruments, especially those previously 
demonstrated. The suggested time for training is May, June, or August. 

• Have every program represented at the training sessions, especially those staff (or 
others) who are conducting assessment. Make certain space is available. The 
preferred method is to allow observations of assessments being conducted in a 
natural or simulated environment. However, quality videos would be quite useful 
for this purpose. 

• Offer intensive training in administering specific assessment instruments as 
opposed to the general training that CIRCLE offered in Summer 2000. 



r 
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